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(g) Digital image generation. 

Q) A method of generating a ret^sentation of a 
^ subsidiary coloured image initially contorted 
within a larger image represented by digital 
data defining the colour content of pixels of the 
image comprises defining a control data array. 
This control data array (9) defines for at least 
those pixels in the image corresponding to the 
region containing fee subsidiary image, the 
proportion of the image pixel colour which is 
due to the subsidiary image. The average colour 
in regions of the image different from the sub- 
sidiary image region is then determined. The 
inverse of the control data array is multiplied by 
the determined, average colour to generate a 
background image.- (13) which is subtracted (14) 
from the corresponding region of the original 
image so as to generate the representation of 
the subsidiary image. 
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DIGITAL IMAGE GENERATION 



The invention relates to methods and apparatus 
for generating a representation of a . subsidiary col- 
oured image initially contained within a larger image 
represented by digital data defining the colour content 
of pixels of the image. 

In the field of digital image processing, there is a 
common requirement to be able to extract a feature 
from an image so that the feature can be printed by 
itself or possibly introduced into another image. For 
example, an original image may have a foreground 
feature which it is desired to extract from the sur- 
rounding background. This is conventionally 
achieved by generating an array of control data or 
mask which matches the original image pixel by pixel, 
each pixel of the control data array defining whether 
the corresponding pixel in the original image is to be 
selected or not More recently, it has been proposed 
to make use of so-called "soft* masks in which each 
pixel in the control data array defines the proportion 
of the corresponding image colour which is to be 
selected. However, even with this recent improve- 
ment, certain subsidiary images are difficult to extract 
For example, in the case of a subsidiary image having 
very fine detail (such as human hair) there will be 
small regions of background intermixed with the fine 
detail which up to now could not be eliminated. 

EP-A-0070174 describes a digital chroma-key 
apparatus for improving the correspondence of the 
keying signal with desired boundary between an 
object and the background. A background colour is 
modulated by a keying signal which has been derived 
by comparing a foreground colour video data signal 
with the background colour data signal and then pro- 
cessed to adjust the timing or compensate the level of 
the digital key signal. 

in accordance with one aspect of the present 
invention, a method of generating a representation of 
a subsidiary coloured image initially contained within 
a larger image represented by digital data defining the 
colour content of pixels of the image comprises defi- 
ning a control data array which defines for at least 
those pixels in the image corresponding to the region 
containing the subsidiary image the proportion of the 
image pixel colour which is due to the subsidiary 
image ; determining the average colour in regions of 
the image different from the subsidiary image region; 
multiplying the inverse of the control data array by the 
determined average colour to generate a background 
image ; and subtracting the background image from 
the corresponding region of the original image so as 
to generate the representation of the subsidiary 
image. 

In accordance with a second aspect of the pre- 
sent invention, apparatus for generating a represen- 
tation of a subsidiary coloured image initially 



contained within a larger image represented by digital 
data defining the colour content of pixels of the image 
comprises first means for generating and storing a 
5 control data array which defines for at least those 
pixels in the image correpsonding to the region con- 
taining the subsidiary image the proportion of the 
image pixel colour which is due to the subsidiary 
image ; second means for determining the average 
10 colour in regions of the image different from the sub- 
sidiary image region ; multiplying means for multiply- 
ing the inverse of the control data array by the 
determined average colour to generate a background 
image ; and subtraction means for subtracting the 
15 background image from the corresponding region of 
the original image so as to generate the represen- 
tation of the subsidiary image. 

We have realized that it can generally be 
assumed that in those areas of the subsidiary image 
20 to be extracted corresponding to "soft? areas of the 
mask or control data array, the actual original image 
colour is equal to : 

(wanted area colour) w (Soft mask value (density)) 
+ (unwanted colour)*(l -soft mask value (density)} 
25 The method then determines 

(Original image colour) - (Average unwanted col- 
our) * (I - soft mask density) 
which is approximately : 
(wanted area colour) * (Soft mask density) 
30 Consequently, the unwanted colour has been 
removed. 

In the simplest case, the invention enables a 
foreground feature to be extracted from an image hav- 
ing a single background colour. More commonly, how- 
35 ever, there may be a series of different background 
colours so that a corresponding number of average 
colours are determined and corresponding areas of 
the inverse of the control data array are multiplied by 
the respective average colour. 
40 The invention is particularly applicable to extract- 

ing a subsidiary image and then inserting that sub- 
sidiary image into another image. In one example, this 
can be achieved by multiplying data defining the col- 
our content of pixels of another image by the inverse 
45 of the control data array and thereafter combining the 
representation of the subsidiary image. 

The combining step may comprise a simple addi- 
tion although in practice a mora complex blending 
may be necessary so as to ensure that the join be- 
so tween the new image and the subsidiary image is 
"seamless". It may also be necessary to compute 
image noise distributions in the two images and add 
noise as appropriate again to ensure a seamless join. 
For example, unwanted noise and other high spatial 
55 frequencies could be removed by suitable choice of 
mixing function, when the image is applied on the new 
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background. The mixing function would be derived 
from the soft mask density. Typically, this could be 
done by ampl^ying the soft mask function {far 
example by multiplying by a scale factor) and then 
lirnitSng.tne results to 100%. The wanted image would 
be multiplied fay this new mask value, before being 
added to the new background image multiplied by the 
old niasjc function. 

The first and second means, the multiplying 
means and the subtracting means may be provided by 
hardware components but will typically be constituted 
by a suitably programmed computer. For example, 
the method could be implemented on the Crosfield 
Studio Image Retouching System. 

Where the images are not monochrome, the 
method will be repeated for each colour component 
An example of a method and apparatus according to 
the invention will now be described with reference to 
the acccmr^anying drawings, in which > 

Figure 1 is a block diagram of the apparatus ; 
Figure 2 is a flow diagram illustrating the oper- 
ation of the apparatus shown in Figure 1 ; 
Figure 3 illustrates an image ; and, 
Figures 4A-4G illustrate the variation of colour 
content and mask content across a portion of the 
image shown in Figure 3, as is explained below. 
The apparatus has a generally conventional form 
and comprises a processor 1 to which commands are 
fed by an operator using a digitising tablet 2 and cur- 
sor 3. Digital data defining the colour content of pixels 
of images is stored in a store 4 and digital data defin- 
ing masks is stored in a mask store 5. The processor 
1 is coupled to a monitor 6 to enable images stored in 
the store 4 to be viewed. 

In this example, it is intended to describe how an 
image of the form shown in Figure 3 is processed so 
as to extract the foreground image of head and shoul- 
ders 7 from the background blue sky 11 and green 
grass 12. The problem with achieving this extraction 
lies in the fine detail of the hair region 7 where small 
portions of the blue sky can be viewed. Clearly, it will 
be very undesirable if the extracted image is inserted 
back into an image with a differently coloured back- 
ground for the old background still to be visible. . 

In this example it is assumed that the hair colour 
is on average C = 60%, M = 80%, Y = 80% and the 
sky colour is on average C = 40%. M = 0%, Y = 0% 
where C is cyan, M is magenta and Y is yellow. Figure 
4A illustrates therefore the variation In cyan in the hair 
region. Figure 4B illustrates the corresponding vari- 
ation in magenta. The variation in yellow is similar to 
4B, so is not shown. Thus, pixels which solely define 
the hair image wiB have a cyan content of 60% and a 
magenta content of 80% while pixels defining solely 
the sky background region w31 have a cyan content of 
40% and a magenta content of 0% (with some noise). 
There will, however, be some pixels in the image 
where neither pure hair nor pure sky colour is present 



and this is indicated by the region noted as "mixture" 
in Figures 4A and 4B. 

In order to extract the foreground image shown in 
Figure 3, the operator uses the digitizing tablet 2 and 

5 cursor 3 to control the processor 1 to carry out the fol- 
lowing steps as illustrated in Figures 2 and 3. 

To extract the foreground image shown in Figure 
3, the operator intialry defines a soft mask This may 
be defined using a conventional technique or thetech- 

10 nique described in our co-pending patent application 
entitled "Improvements Relating to Control Data 
Arrays" (Agent's Ref : 30/2997/01). This soft mask 
comprises an array of control data pixels, there being 
one control data pixel corresponding to each original 

15 image pixel. The content of each control data pixel 
defines the proportion of the colour components in the 
corresponding pixel of the original image which is to 
be extracted. Thus, for the hair/sky region represen- 
ted in Figures 4A and 4B, the soft mask values will 

20 have the form shown in Figure 4C. This indicates that 
for image pixels solely representing hair it is desired 
to extract 100% of that jpbcel data while for image 
pixels representing solely sky (Le. undesired back- 
ground) it is desired to extract 0%. There is then a 

25 general gradation in the : mixture region. The mask 
definition step is indicated by step 8 in Figure 2. The 
mask is stored in the mask store 6. 

in a step 9 the inverse of the stored mask is deter- 
mined. Where the mask values or densities are nor- 

30 malised to vary between 0 and 1 then the inverse 
mask is defined as, I-mask value. This inverse mask 
is stored in the store 5. 

In a step 10 the processor 1 calculates the aver- 
age background colours in the image. In the case of 

35 the Figure 3 image, there are two background col- 
ours : the blue sky region 1 1 and the green grass reg- 
ion 12. 

Each pixel in the inverse mask previously deter- 
miriedis then multiplied by the corresponding average 

40 background colour and the resultant background 
image is stored in the image store 4 (step 13). This 
background image effectively defines, in the region of 
the image to be extracted, the amount of undesired 
background colour. Figure 4D illustrates the variation 

45 of the undesired background cyan in the hair/sky reg- 
ions where it will be seen that for image pixels defining 
solely hair then there is no undesired cyan back- 
ground whereas in regions defining solely sky the 
entire cyan (40%) is undesired whfle a general gra- 

50 dation takes place in the mixture region. For magenta 
and yellow the variation is shown in Figure 4E 

The processor 1 then subtracts the background 
image generated in step 13 from the original image so 
as to generate the extracted image (step 14). This 

55 effectively corresponds in the case of the cyan colour 
component to the subtraction of the trace in Figure 4D 
(undesired background cyan) from the trace in Fgure 
4A (original image cyan) and in the case of magenta « 
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and yellow subtraction of the trace in Figure 4E (unde- 
sired background magenta oryefiow) from that in Fig- 
ure 4B (original image magenta or yeOovv). The 
resultant variations in cyan, and magenta or yellow 
are shown in Rgures 4F and 4G respectively, H will be 
seen from these latter traces that in thcs mixture region 
the colour component content falls to 0 for all colour . 
components. * 

This extracted image can then be displayed after 
conversion to red, green, blue components on the 
monitor 6 (step 1 5} to allow the operator to check that 
the extraction has worked successfully (step 15). The 
extracted image is then stored in the image store 4. 

If desired, the operator can arrange for the extrac- 
ted image to be inserted into a new background image 
also in the store 4. This is achieved by multiplying the 
new background image by the previously determined 
inverse mask (step 16) and then adding the extracted 
image (step 17). The resultant can then be displayed 
on the monitor 6 (step 18). 

As has been mentioned above, instead of the sim- 
ple addition of step 17, a more complex blending may 
be necessary with a possible addition of noise to 
ensure a seamless join in between the images. 
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Claims 

1 . A method of generating a representation of a sub- 
sidiary coloured image initially contained within a 30 
larger image represented by digital data defining 
the colour content of pixels of the image, the 
method comprising defining a control data array 
which defines foratleast those pixels in the image 
corresponding to the region containing the sub- 35 
sidiary image the proportion of the image pixel 
colour which is due to the subsidiary image (9) ; 
determining the average colour in regions of the 

image different from the subsidiary image region 

(10) ; multiplying the inverse of . the control data 40 
array by the determined average colour to gener- 
ate a background image (1 3) ; and subtracting the 
. background image from the corresponding region 
of the original image so as to generate the repre- 
sentation of the subsidiary image (14). 45 



Apparatus for generating a representation of a 
subsidiary coloured image initially contained 
within a larger image represented by digital data 
defining the colour content of pixels of the Image, 
the apparatus comprising first means (4) for 
generating and storing a control data array which 
defines for at least those pixels in the image cor- 
responding to the region containing the sub- 
sidiary image the proportion of the image pixel 
colour which is due to the subsidiary image ; sec- 
ond means for determining the average colour in 
regions of the image different from the subsidiary 
image region ; multiplying means for multiplying 
the inverse of the control data array by the deter- 
mined average colour to generate a background 
image ; and subtraction means for subtracting the 
background image from the corresponding region 
of the original image so as to generate the repre- 
sentation of the subsidiary image. 



A method according to claim 1 for inserting the 
subsidiary image into another image, the method 
comprising multiplying data defining the colour 
content of pixels of the another image by the 
inverse of the control data array and thereafter 
combining the representation of the subsidiary 
image. 



so 



A method according to claim 2, wherein the com- 
bining step comprises adding the representation 
of the subsidiary image to the multiplied data. 
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